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ABSTRACT 

This study examined practices of teachers in Wake County Public Schools’ (WCPSS) 
middle school Algebra I classes. Regression analyses of standardized state testing 
results allowed for identification of the most effective and least effective Algebra I 
teachers. The study used surveys, observations, and focus group interviews to 
compare and contrast most effective teachers with less effective teachers. It found 
that the most effective middle school algebra teachers held a significantly more 
positive attitude toward their students than did less effective teachers. They had high 
expectations for all students, used mathematics vocabulary appropriately, had a 
structured classroom management style, taught bell to bell using an invigorated pace, 
and had a classroom culture in which students were free to ask questions, contribute, 
and offer explanations. These results can be used to motivate teacher and school 
improvement efforts. 


INTRODUCTION 

The importance of algebra in preparing students to be successful and productive citizens of the 
2F* century is now widely, if not completely, accepted. Moses (1995) named algebra “the new 
civil right.” In Principles and Standards for School Mathematics, there is an algebra standard in 
all grade spans from Pre-K to 12. At the middle school level, 
grades 6 to 8, the National Council of Teachers of Mathematics 
(NCTM) proposes that students “learn significant amounts of 
algebra” (National Council of Teachers of Mathematics, Inc. 

[NCTM], 2000, p. 212). 
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Despite this, there are still many areas of disagreement about the delivery of algebra. Current 
debates address three main topics: the definition of algebra, the content of an Algebra I class, and 
when should students first take Algebra I. Steen (1999) argued that there were “virtually 
insurmountable impediments” to all students studying algebra in eighth grade. The barriers that 
he identified included unprepared students, lack of readiness for the abstraction of algebra, a 
shortage of middle school teachers prepared to teach algebra, a lack of course materials, 
including textbooks, that appropriately bridged the transition from arithmetic to algebra, and 
teacher beliefs that most students cannot learn algebra in eighth grade. Despite these barriers, 
there has been a movement to require algebra of all eighth grade students. In 2008, California 
made algebra mandatory in eighth grade. However on October 28, 2008, a California Superior 
Court judge ordered the state Board of Education to stop all further plans until a December 19 
court hearing. Proponents of eighth grade algebra argue that even low performing students learn 
more in an algebra course than in an eighth grade mathematics course, and that all students 
should have the opportunity to prepare to study advanced mathematics in high school (Mathews, 
2008a, 2008b; Williams, 2008). 

Those opposing eighth grade algebra for all are supported by a recent study of the Brookings 
Institution that found some students scoring in the bottom 10* percentile on the National 
Assessment of Educational Progress (NAEP) eighth grade mathematics test who were also taking 
Algebra I or even geometry. The conclusion of this report is that there are many students in 
algebra who are misplaced. These students lack basic preparation and often turn the Algebra I 
class into a basic arithmetic skills class (Eoveless, 2008). NCTM recently released a position 
paper that summarizes their stance on these controversial algebra discussions. It includes the 
following statement: “Exposing students to such coursework before they are ready often leads to 
frustration, failure, and negative attitudes toward mathematics and learning” (NCTM, 2008). 

ALGEBRA IN NORTH CAROLINA 

The North Carolina Standard Course of Study (NCSCS) reflects the NCTM algebra standard by 
incorporating algebraic concepts from Kindergarten to 12* grade, and increasing amounts of 
algebra in grades six to eight (20%-25% of the curriculum in grade 6, 25%-30% in grade 7, and 
35% to 40% in grade 8). The majority of the concepts regarding linear equations from the 
Algebra I 1998 NCSCS were moved to the eighth grade math curriculum in the 2003 NCSCS. 
Students study evaluating algebraic expressions, solving simple equations and inequalities, and 
solving problems involving rates of change in sixth grade; linear relationships and fundamental 
algebraic concepts in seventh grade; and detailed work with linear relationships including finding 
slope and equations of lines in eighth grade (North Carolina Department of Public Instruction 
[NCDPI], 1998, 2003a, 2003b, 2003c). 

The official North Carolina Algebra I course of study reflects all the prior algebra goals in grades 
6 to 8 as prerequisites. Although most students take Algebra I in high school, there are many 
academically advanced middle school students who master the mathematics goals of grades 6 to 
8 before eighth grade (a rare few before sixth or seventh grades), and proceed to take the Algebra 
I course in middle school. 
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ALGEBRA I AND THE STATE ACCOUNTABILITY MODEL 

In North Carolina, End-of-Course (EOC) exams are administered in 10 high sehool courses, 
including Algebra I. Each exam is a standardized multiple-choice test written with input from 
teachers across the state. Teachers participate in test development in a variety of ways, from 
writing the curriculum, on which EOC tests are based, to writing and reviewing test items. Each 
student who takes an EOC test is assigned a scale score based on the number of items correct and 
the difficulty of items. The scale scores are then converted to one of four levels of performance. 
Eevels III and IV are associated with adequate or higher mastery of course content, and are 
considered proficient (NCDPI, 2007). The percentages of students passing each EOC in a school 
are reported publicly, and teachers often judge their own success using these percentages. 

Eor middle schools, the Algebra I EOC tests administered each year are a small component of 
the ABCs of Public Education, the state’s accountability program. The program actually has two 
measurement standards: the absolute percentage of tests at or above grade-level proficiency, and 
the attainment of “expected” growth. The basic assumption of the growth part of the model is 
that a student should be expected to do at least as well on each EOC test as the student’s prior 
performance on End-of-Grade (EOG) and EOC tests. Teachers with high-achieving students do 
not always produce expected growth in their students. The expected growth measure is 
considered by most teachers to be a fairer measure of success than student proficiency alone, 
because it takes into account the skill set that students bring to the course (NCPDI, 2008). 

WCPSS MIDDLE SCHOOL ALGEBRA I 

Demographics 

In 2007-08, 3,360 students (29% of all WCPSS Algebra I students) took Algebra I in middle 
school. Most of these students, 3,097, were in eighth grade. Most WCPSS middle school 
students prepare for Algebra I by taking a compacted mathematics course (Pre -Algebra) that 
covers both the seventh and the eighth grade mathematics curriculum during their l'^ grade year. 
Thirty percent of 2007-08 eighth grade students took Algebra I as their mathematics course. 
There were also five sixth grade and 258 seventh grade Algebra I students in 2007-08. By the 
end of 2007-08, 33% of WCPSS eighth grade students had taken Algebra I. This is the same 
percentage as North Carolina as a whole, but a little lower than the national average of 38% 
(Loveless, 2008). 

The ethnic enrollment of 2007-08 middle school Algebra I was 69.8% White, 10.1% Asian, and 
16.5% Black/ African American or Hispanic/Latino. There were slightly more males than 
females (50.5% compared to 49.5%). The overall 8* grade mathematics enrollment was 52.4% 
White, 5.3% Asian, and 38.0% Black/ African American or Hispanic/Latino with 51.5% male 
and 48.5% female (Table 1). 
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Table 1 

WCPSS Middle School Algebra I Enrollment by Gender and Ethnicity 

2007-08 



Aigebra I 

8th grade Math 


Maie 

Femaie 

Totai 

Maie 

Femaie 

Totai 


n 

%of 

course 

n 

%of 

course 

n 

%of 

course 

n 

%of 

course 

n 

%of 

course 

n 

%of 

course 

American Indian 

4 

0.1% 

2 

0.1% 

6 

0.2% 

18 

0.2% 

12 

0.1% 

30 

0.3% 

Asian 

173 

5.1% 

167 

5.0% 

340 

10.1% 

289 

2.9% 

239 

2.4% 

528 

5.3% 

Biack/African American 

160 

4.8% 

245 

7.3% 

405 

12.1% 

1419 

14.3% 

1403 

14.1% 

2822 

28.4% 

Hispanic/Latino 

75 

2.2% 

72 

2.1% 

147 

4.4% 

483 

4.9% 

470 

4.7% 

953 

9.6% 

Multiraciai 

65 

1.9% 

53 

1.6% 

118 

3.5% 

208 

2.1% 

186 

1.9% 

394 

4.0% 

White 

1221 

36.3% 

1123 

33.4% 

2344 

69.8% 

2697 

27.2% 

2503 

25.2% 

5200 

52.4% 

Ali Students 

1698 

50.5% 

1662 

49.5% 

3360 

100.0% 

5114 

51.5% 

4813 

48.5% 

9927 

100.0% 


Student Performance and Growth 


Middle school Algebra I students are good students who perform well and out perform their high 
school peers. In 2007-08, 97% of middle school Algebra I students scored at Level III or IV (the 
proficiency levels) on the EOC exam (Asian, 99%; Black/African American or Hispanic/Latino, 
88%; and White, 98%). Seventy-nine percent of these students scored at Level IV (Asian, 91%; 
Black/African American or Hispanic/Latino, 50%; and White, 84%). 

The distribution of scale scores is slightly skewed left with a mean of 1 64 compared to the high 
school distribution that is more bell-shaped with a mean of 152 (Figures 1 - 3). 


Figure 1 

Level Performance of 2007-08 Middle School Algebra I Students 
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■ Level IV 

5 

308 

203 

73 

95 

1962 

2646 

■ Level III 

1 

30 

145 

62 

21 

344 

603 

■ Level II 

0 

2 

50 

10 

2 

34 

98 

■ Level 1 
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7 

2 

0 

4 
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Figure 2 

2007-08 Distribution of Middle School Algebra I Scale Scores 



Figure 3 

Comparison of 2007-08 Middle School and High School Scale Score Distributions 



Scale Scores 
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As shown in Table 2, percentages of proficient students in 2007-08 ranged from 84.5% to 100%. 
Of 28 middle schools, the percentages of students who met their North Carolina expected growth 
target ranged from 21.7% to 89.2%. All but nine schools had 60% or more, thus making high 
growth. Yet six of the nine schools with less than 60% did not make expected growth either. 
Moreover, four of the schools that had more than 90% of their students proficient did not meet 
high growth. The growth percentages reveal an area where improvement can be made. To reach 
the WCPSS goal of “all students demonstrating high academic growth and graduating on time 
prepared to compete globally”, schools need to set the goal of all Algebra I students maximizing 
their performance, not just reaching minimum proficiency (Table 2). Note that two new middle 
schools had only a sixth grade, and did not offer Algebra I in 2007-08. Three other middle 
schools were alternative schools and are also excluded here. 


Table 2 

2007-08 Performance and Growth by Middle School 


School 

Proficiency School Rank 

% Proficient 


■ill.I.ldMMd.l 

Growth School Rank 

A 

1 

100.0 

89.2 

met 

1 

B 

1 

100.0 

86.4 

met 

2 

C 

1 

100.0 

79.9 

met 

5 

D 

1 

100.0 

78.5 

met 

7 

E 

1 

100.0 

74.4 

met 

9 

F 

1 

100.0 

72.2 

met 

12 

G 

7 

99.4 

84.2 

met 

3 

H 

8 

99.3 

66.9 

met 

15 

1 

9 

99.2 

81.2 

met 

4 

J 

10 

99.1 

76.3 

met 

8 

K 

11 

98.6 

72.9 

met 

11 

L 

11 

98.6 

65.4 

met 

16 


11 

98.6 

61.6 

met 

19 

N 

11 

98.6 

43.1 

not met* 

26 

0 

15 

97.7 

73.4 

met 

10 

P 

16 

97.1 

64.7 

met 

17 

Q 

17 

96.9 

79.6 

met 

6 

R 

18 

96.7 

64.4 

met 

18 

S 

19 

96.4 

57.7 

not met 

21 

T 

20 

95.0 

67.1 

met 

14 

U 

20 

95.0 

48.2 

not met* 

24 

V 

22 

92.5 

52.9 

not met 

22 

w 

23 

92.4 

68.2 

met 

13 

X 

24 

89.9 

58.7 

not met 

20 

Y 

25 

89.3 

45.6 

not met* 

25 

z 

26 

88.0 

21.7 

not met* 

28 

AA 

27 

87.0 

41.2 

not met* 

27 

BB 

28 

84.5 

48.4 

not met* 

23 


* did not make expected growth 
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IMPORTANCE OF TEACHER QUALITY TO STUDENT SUCCESS 

The performance of students on the standardized EOG and EOC exams is not only used in the 
North Carolina ABCs but also as the main measure of the success of the federal Elementary and 
Secondary Education Act of 1965 (ESEA), amended by the No Child Left Behind Act of 2001 
(NCLB). Several studies of student gains on standardized tests from one year to another have 
found the student’s assigned teacher to be the most influential factor (Rivkin, Hanushek, & Kain, 
2001; Sanders & Horn, 1994; Sanders & Rivers, 1996; Wright, Horn, & Sanders, 1997). 

Over the past twenty years, the importance of the classroom teacher has emerged as a key 
component of school reform. In 1987, the National Board for Professional Teaching Standards 
was established with a mission of advancing the quality of learning by advancing the quality of 
teaching. In 1989, the National Board issued a policy statement. What Teachers Should Know 
and Be Able to Do (National Board for Professional Teaching Standards [NBPTS], 1989), which 
posited five core propositions to guide the certification of National Board Teachers: 

• commitment to students and learning, 

• knowledge of the subject taught and effective methods of teaching it, 

• responsibility for managing and monitoring student learning, 

• systematic reflection on practice that leads to improved practice, and 

• membership in a learning community. 

NCLB also emphasizes teacher quality (U.S. Congress, 2001). Under NCLB, every state must 
develop and implement a plan to insure that all students will be taught by a “highly qualified 
teacher” (HQT; sec. 2101). Margaret Spellings, U.S. Secretary of Education, stated in a letter 
dated October 21, 2005, “There is also evidence that states are improving the quality of their 
teaching forces.” She wrote that the U.S. Department of Education was committed to the goal of 
every child being taught by an HQT by the end of 2005-06, and then outlined the actions that 
would be taken to support states in reaching this goal (Spellings, 2005). The NCLB law requires 
that all academic core subject teachers be highly qualified. The NCLB law (U.S. Department of 
Education, 2006) uses three key guidelines to determine whether a teacher is highly qualified: 

• at least a bachelor’s degree in the subject taught, 

• full state teacher certification, and 

• demonstrated knowledge in the subject taught. 

The importance of teachers is recognized by national subject-specific professional teaching 
organizations. NCTM in Principles and Standards for School Mathematics, details requirements 
of effective teaching, including the requirement to continually seek improvement. “The 
improvement of mathematics education for all students requires effective mathematics teaching 
in all classrooms” (NCTM, 2000, p. 17). 

MIDDLE SCHOOL ALGEBRA TEACHERS 

Steen’s (1999) concern that there is a shortage of middle school teachers prepared to teach 
algebra was supported in 2007 by a new study conducted by researchers from Michigan State 
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University funded by a National Seience Foundation Grant. Led by William H. Sehmidt, the 
research found that United States middle school mathematics teachers were not as well-prepared 
to teach mathematics as middle school teachers in five other countries (South Korea, Taiwan, 
Germany, Bulgaria, and Mexico). Algebra knowledge was particularly lacking with prospective 
U.S. teachers scoring at the bottom of all countries in the study (a full standard deviation below 
Taiwan). In the U.S., middle school mathematics teachers are prepared in one of three programs 
(secondary, middle, or elementary). Teachers trained in a secondary program were strongest in 
content with those from elementary programs strongest in pedagogy. The middle school 
programs were weaker in both areas. Teachers from elementary or middle school programs were 
more likely to view mathematics as algorithmic than teachers from a secondary preparation 
(Schmidt et al, 2007). 

In Arkansas, where middle school teachers are certified with a general certificate across subjects 
for grades 4 to 8, middle school teachers must now earn a state endorsement in algebra to teach 
Algebra I . The new requirement can be met after completion of a 15-hour program of advanced 
mathematics study including calculus and a minimum score on the Praxis II for middle school 
mathematics. Until now, Arkansas has only allowed teachers with a high school mathematics 
license to teach middle school Algebra I. Some middle school Algebra I students had to travel to 
a high school or be taught by a borrowed high school teacher (Cavanagh, 2008). 

In North Carolina, a mathematics middle grades licensure includes Algebra I. Licensed teachers 
have completed 24 hours of mathematics, successfully passed the Math Praxis, and graduated 
from a recognized college or university. 

MEASUREMENT OF TEACHER QUALITY 

Student standardized test performance has been a commonly used measure of teacher quality. In 
North Carolina, the state provides to every school district test analysis software that can be used 
to run school-level results and results for subgroups of students within the school. The state also 
posts these disaggregated test results on a web site. These analyses, however, are limited to 
average scale scores, percentages of students tested who attain proficiency, and state growth 
measures. Although these state-provided measures serve as a valid way of reporting how 
teachers and schools are succeeding, the WCPSS Evaluation and Research department has 
developed more fine-grained methods to determine which schools and teachers are getting the 
most growth with students in comparison to other WCPSS students. Identifying the schools and 
teachers that are getting the highest performance from students at varying levels of preparation is 
necessary in order to be able to share best practices within the district and motivate school 
improvement efforts. 

WCPSS Residuals and Effectiveness Index 

Since the early 1990s, WCPSS has used a multiple regression analysis to generate an 
“effectiveness index” that measures achievement at each school within WCPSS. The regression 
analysis creates a prediction model by using the current year’s test scores as the outcome and 
previous years’ test scores as the predictors. The analysis also takes into account each student’s 
special program status level of service (e.g. self-contained), free or reduced-price lunch (ERL) 
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status, and academically gifted (AG) status as well as the percentage of FRL students in the 
school. A residual score is calculated for each WCPSS student who took the test and had the 
necessary previous test scores to use as predictors. The residual score for a student is the 
difference between the student’s actual score and the score that the regression analysis model 
predicted. These residuals measure how students performed compared to other students in 
WCPSS with similar previous test scores and program identifiers. 

As an example, Figure 4 gives a simplified visual of the regression analysis for Algebra I with 
three of the many possible lines. It shows that the predictions are both dependent on previous 
FOG and/or EOC scores and also the student’s program characteristics, such as AG or a student 
with a disability (SWD) in a resource setting who is also FRL. The middle line could represent 
students with no identifiers (e.g., typical students). 


Figure 4 

Algebra I Regression Scatter Plot 
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Figure 5 shows one vertieal sliee of the regression model for students whose predietion line is the 
middle line of Figure 4 and whose previous math FOG score was 170. Therefore, they share the 
exact profile on all of the factors used to predict the outcome. The line at the center is the 
predicted score generated by the model for these students. A residual is the difference between 
the actual score and the predicted score. Scores above the red line have positive residuals and 
scores below have negative residuals. 


Figure 5 

Student Residuals for a Hypothetical Group of Students 


Algebra scale score 75, residual 75-60 = 15 


Algebra scale score 60 ► 







These scores have positive residuals. 




■This is the predicted score from the regression 
model. 


These scores have negative residuals. 


Previous Math EOG score of 170 


For each test given, the residuals are averaged across all students in the school, and a 
standardized z-score (effectiveness index) is generated for each school by subject. A z-score is 
the number of standard deviations that the school’s residual average is from the average (mean) 
of all the schools with at least 30 students who have test scores and previous scores. If the z- 
score is greater than 1, then the school knows that its students in that course have scored 
significantly higher (among the top 16%) than other students in the district after taking into 
consideration their previous test scores and program characteristics. Similarly, if the 
effectiveness index is less than -1, then the students have scores much lower (among the bottom 
16%) than other students in other schools. Values between -1 and +1 are within one standard 
deviation of the district average and are considered “typical” or expected (68% of scores should 
be in this range). 

In addition to the school’s z-scores, WCPSS principals receive rosters of student residuals by 
teacher, course, and section. In these rosters, student residuals above one standard deviation are 
coded in green, and student residuals below one standard deviation are coded in red. The 
standard deviation in scale-score points of these residual scores is displayed at the bottom of 
roster, along with the average residual for the section. 

Table 3 is a sample roster for a 2006-07 middle school Algebra I class of 18 students. For each 
student, the predictor scores are shown. The predictors for the Algebra I EOC were the 7th- 
grade reading and mathematics scores. The roster then displays the Algebra I EOC scale score 
and the residual score for each student. These residuals are averaged and an average residual 
score for the class is provided. The average residual for this class was 0.30. The principal and 
teacher can then determine how successful students were on the EOC as compared with other 
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students with similar characteristics. Notice that Student 8, Student 12, and Student 16 have the 
same scale score on the Algebra I EOC exam, but Student 8 has a negative residual while 
Student 12 and Student 16 have positive residuals. Student 16 has lower scale scores on the 7*'^ 
grade EOG exams than Student 8 thus explaining the much higher residual. The residual shows 
a measure of performance as related to previous performance and other educational indicators, 
and gives a sense of the relative growth for each student. Student 1 has no residual, as this 
student is missing previous test scores required to calculate a residual. 

Table 3 

Sample Middle School Algebra I EOC Residual Roster 


Name 

7‘*'-Grade EOG 
Reading Scale 
Score 

7'*’-Grade EOG 
Math Scale 
Score 

2007 

A1 

Scale score 

2007 

A1 

Residual 

Student 1 

• 

• 

179 

• 

Student 2 

264 

366 

152 

-7.74 

Student 3 

262 

370 

158 

-5.77 

Student 4 

277 

370 

163 

-3.07 

Student 5 

268 

372 

162 

-2.60 

Student 6 

276 

375 

167 

-2.29 

Student 7 

269 

369 

162 

-2.02 

Student 8 

271 

376 

168 

-1.32 

Student 9 

274 

379 

171 

-1.12 

Student 10 

272 

371 

164 

-0.43 

Student 1 1 

281 

382 

175 

-0.24 

Student 12 

272 

373 

168 

0.46 

Student 13 

267 

372 

165 

1.57 

Student 14 

274 

374 

170 

3.14 

Student 15 

281 

372 

170 

3.61 

Student 16 

268 

368 

168 

6.28 

Student 17 

267 

374 

173 

7.11 

Student 18 

267 

370 

174 

9.54 


Note: Class Average = 0.30 

Standard deviation = 5.09 

Interpretation Example: Student 16 had a 7*-grade reading scale score of 268 and a 7*-grade mathematics scale 
score of 368. Student 16’s 2007 Algebra I scale score was 168 and the student’s residual was 6.28. 
This student scored among the top 16% of students with the same Algebra I score, the same 7*-grade 
mathematics scores and the same academic program indicators. 


WCPSS Middle School Effectiveness 

Table 4 shows the Algebra I effectiveness indices of 26 WCPSS middle schools in 2004-05, 
2005-06, 2006-07, and 2007-08 (two of the previous 28 schools were new and did not have data 
for three prior years). Schools with an effectiveness index above 1 are coded as H for high and 
below a -1 as E for low. Schools with effectiveness indices between -1 and 1 are coded with an 
M. The schools were ordered with the same identifying letter as in Table 2, thus in order from 
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highest percentage proficient to lowest. Since middle schools have an average of three to four 
Algebra I teachers per school, the performance of each teacher contributes significantly to the 
overall school effectiveness. 


Table 4 

School Level WCPSS Effectiveness Indices 


School 

04-05 

05-06 

06-07 

07-08 

A 

M- 

H 

M+ 

H 

B 

L 

H 

M+ 

M+ 

C 

M- 

M+ 

M- 

M- 

D 

M- 

L 

L 

M+ 

F 

H 

H 

M- 

H 

G 

M+ 

H 

M+ 

M+ 

H 

M+ 

M+ 

M+ 

M- 

1 

L 

L 

M- 

M- 

J 

H 

H 

H 

M+ 

K 

H 

H 

M- 

M+ 

L 

M- 

M+ 

M+ 

M- 

M 

L 

M+ 

M- 

M+ 

N 

M- 

M+ 

H 

L 

0 

M+ 

H 

H 

M+ 

P 

H 

L 

H 

H 

Q 

L 

L 

M- 

M+ 

R 

L 

L 

M- 

M- 

T 

M- 

L 

L 

M+ 

U 

M+ 

M+ 

L 

L 

V 

L 

L 

M- 

M- 

w 

M+ 

M+ 

M- 

M+ 

X 

H 

L 

M- 

M+ 

Y 

H 

H 

H 

M- 

z 

L 

L 

M+ 

L 

AA 

L 

M- 

L 

M- 

BB 

L 

M+ 

M- 

M- 


Note: 


1. H = effectiveness index > 1 

2. L = effectiveness index < -1 

3. M+ = effectiveness index between 0 and 1 

4. M- = effectiveness index between -1 and 0 
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PURPOSE OF STUDY 

This research study had two main objectives: 

• Study middle school Algebra I using a WCPSS Value-Added Instructional Improvement 
Analysis Model. 

► Collect WCPSS-specific data that will help teachers, schools, and district leadership 
understand their current practices in middle school Algebra I. 

► Identify and share best teaching strategies in middle school Algebra I that are linked to 
high student achievement. 

• Contribute to a series of studies that identify targets for overall systemic improvement. 

► Identify the roles of teachers, academic departments, principals, schools, and central 
services’ administrators in the school improvement process. 

► Identify the practices of effective instruction. 

The classroom practices of the most successful teachers can be documented to challenge teachers 
of middle school Algebra I students to even higher academic goals. Teacher performance 
evaluation was not a goal of this study, unlike most current valued- added models (Braun, 2005; 
Olson, 2005; Olson, 2004a, 2004b; Sanders, 1998; Tucker & Stronge, 2005). This study 
demonstrates the use of value-added research for teacher and school improvement rather than for 
purposes of teacher evaluation. 


METHOD 

Success in this study was defined by using the residuals generated from the WCPSS 
effectiveness study of the state Algebra I EOC standardized tests. The study brought value- 
added meaning to the middle school Algebra I EOC test performance. The student residual 
scores and the effectiveness indices give the district a comparison basis for schools and students. 
Until recently, residuals had not been averaged or standardized at the teacher level beyond the 
classroom roster. Prior to 2005-06, teachers were encouraged to study their rosters for trends in 
student performance, and some principals had compared teachers within their school, but no 
districtwide comparisons had been made. A study of biology teaching by WCPSS in 2004-05 
(Haynie, 2006) was a first attempt at identifying the success of teachers, as indicated by average 
residuals, and then to identify the specific aspects of the practice of highly effective and 
relatively less effective teachers in order to isolate teachers’ classroom practices that may be 
associated with high student achievement. Residual averages by teacher were also used in this 
study to identify teachers to observe. 

This study was a collaborative study between the WCPSS Curriculum and Instruction 
Department (C&I) and the Evaluation and Research Department (E&R), with the goal of 
identifying best teaching practices. Since the effectiveness indices and teacher residual averages 
both use residual values that are calculated using student test results, that are known to contain 
error, three consecutive years of data were combined to both reduce the test error and remove 
inexperienced teachers and teachers who taught Algebra I infrequently. Specialists from C&I 
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took the lead in interpreting elassroom observations while a speeialist from E&R took the lead in 
data collection and analysis. 

CONTEXT AND PARTICIPANTS 

This research study took place in WCPSS, a large urban/suburban school district in North 
Carolina. The WCPSS student population is growing rapidly, with an enrollment of 
approximately 134,000 in 2007-08 with 29,975 middle school students. There were 33 middle 
schools in the district, of which eight were year-round, nine were magnet schools, and three were 
alternative. Excluding the alternative middle schools, there were twenty-eight middle schools 
that offered Algebra I in 2007-08. 

The study began in 2006-07 by identifying 36 middle school Algebra I teachers (42% of all 
middle school Algebra I teachers) who had taught Algebra I in 2003-04, 2004-05, and 2005-06 
and were teaching in 2006-07, which made them eligible for this study. Eor these 36 teachers the 
average student residual across all years and classes was calculated. The teachers were ranked 
on teacher effectiveness from highest to lowest using these averages. The teachers with the eight 
highest residual averages were labeled T1-T8 (“top” teachers) and the teachers with the eight 
lowest averages were labeled Bl-B 8 (“bottom” teachers). The practice of these 16 teachers 
became the focus of this study. 

Table 5 shows the residual average (in scale score points), the residual standard deviation, the 
number of students, and the percentage of minority (Black/ African American or Hispanic/Eatino) 
students included in the average. The school code used was randomly assigned and not the same 
as in Tables 2 or 4. Table 5 demonstrates several points: 

• The top 8 teachers were in 8 schools. 

• The bottom 8 teachers were in 6 schools. 

• Residual averages ranged from -4.07 to 5.02. 

• The largest Black/Hispanic minority presence was in T1 ’s classes with 36%, followed by T4, 
B2, B6, and B1 at 19%, 13%, 13%, and 11%. 

• The lowest Black/Hispanic minority presence was in the classes of T8, B4, and B5 at 2% 
each. 
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Table 5 

Residual Averages of Study Teachers 


teacher 

school 

At 

Residual 

Mean 

A1 

Residual 

Standard 

Deviation 

3-Year 
Residual N 

% Black/ 
Hispanic 

T1 

S4 

5.02 

4.85 

140 

36% 

T2 

S9 

4.15 

4.05 

129 

5% 

T3 

S12 

2.90 

5.40 

112 

5% 

T4 

S5 

2.37 

5.55 

119 

19% 

T5 

S2 

2.29 

4.80 

209 

3% 

T6 

S6 

1.79 

4.88 

100 

6% 

T7 

S7 

1.74 

5.05 

96 

5% 

T8 

S13 

1.41 

5.06 

118 

2% 

B8 

S11 

-0.30 

5.64 

121 

5% 

B7 

S14 

-0.46 

4.81 

155 

10% 

B6 

S10 

-1.18 

5.04 

94 

13% 

B5 

S1 

-1.87 

4.98 

101 

2% 

B4 

S1 

-2.37 

4.91 

114 

2% 

B3 

S10 

-2.66 

4.50 

120 

9% 

B2 

S3 

-2.99 

5.86 

103 

13% 

B1 

S8 

-4.07 

4.69 

97 

11% 


Overall the mean of the top teaeher residual averages was 2.77 and for the bottom teaehers it was 
-1.88. The differenee between top and bottom teaeher residual averages was also present when 
the residuals were disaggregated by ethnieity (Table 6). 

Table 6 

Residual Averages by Selected Ethnic Subgroups 


group 

Overall 

Residual 

Mean 

laaSlaM 

White 

Residual 

Mean 

laaSlaM 

Asian 

Residual 

Mean 

maSlaM 

Black 

Residual 

Mean 

Black 

RES_SD 

TOP Teachers 

2.77 

5.07 

2.49 

5.02 

4.5 

4.78 

3.65 

5.81 

BOTTOM Teachers 

-1.88 

5.19 

-1.99 

5.23 

0.82 

4.52 

-2.23 

4.89 


INSTRUMENTS, DATA COLLECTION, AND ANALYSIS PROCEDURES 

In addition to student aehievement seores, four other types of data were eolleeted for this study. 
First, a survey was prepared and distributed to the 36 study teaehers. The teaehers answered 37 
written survey questions eoneeming preparation, planning, use of time, sehedules, use of data, 
and student interaetion. 
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Second, principal investigators for this study made classroom observations of all 16 teachers 
identified as most effective (“top teachers”) or least effective (“bottom teachers”). All but one of 
the 16 teachers were observed by the Evaluation and Research senior administrator. The senior 
administrator from Curriculum and Instruction who began the project left for a new position out- 
of-state before the first scheduled observations in 2006-07. When the position was filled, the 
new senior administrator and the middle school mathematics coordinating teacher made 14 
observations in 2007-08. Two teachers had left WCPSS middle school algebra. The observation 
instrument was adapted from the one previously used in the high school algebra study (Haynie, 
2008). 

Third, teacher focus-group interviews were conducted of the top teachers and the bottom 
teachers. All data collection instruments had some common questions or observations with the 
high school algebra study so comparison would be possible. Fourth, data were collected on 
cohorts of students from the top and bottom middle school and high school teachers as these 
students moved through high school mathematics. 


RESULTS 


STUDENTS 

Prior Academic Achievement 

Prior performance of students on 7th-grade mathematics EOG state tests was averaged for each 
of the top and bottom teachers. Table 7 shows the averages and the standard deviations. For the 
years 2003-06, the overall average 7th-grade mathematics scale score ranged from 278.9 to 288.5 
for top teachers and 279.1 to 287.9 for bottom teachers. The overall average was 284.5 for top 
teachers and 284.0 for bottom teachers. The EOG mathematics test was rescaled in 2005-06. 

Yet for the year 2005-06, scale conversions to the past scale were available and so averaged with 
the previous years. These conversions were not available in 2006-07, so the averages are 
reported on the new scale. The top teachers’ 7*’^-grade scale score average in that year was 
371.7 and the bottom teachers’ average was 370.8. 

The top teacher, Tl, had the lowest previous 7*’'-grade test score averages; closest to the averages 
of teachers Bl, B2, and B7. Teachers T2, B4, and B5 had the highest previous test score 
averages. The overall averages of top and bottom teachers were within 0.5 and 0.9 scale score 
points (Tables 7 and 8). 
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Table 7 

7th-grade Student EOG Mathematics Averages 
2003-2007 


teacher 

school 

Average 
7th grade 
Math EOG 
Scale 
Score 03- 
06 

7th grade 
Math EOG 
Standard 
Deviation 
03-06 

Average 
7th grade 
Math EOG 
Scale 
Score 06- 
07 

7th grade 
Math EOG 
Standard 
Deviation 
06-07 

T1 

S4 

278.9 

7.5 

366.7 

5.5 

T2 

S9 

288.5 

6.2 

373.0 

4.8 

T3 

S12 

286.2 

6.4 

372.4 

4.5 

T4 

S5 

282.3 

7.9 

370.7 

5.6 

T5 

S2 

284.5 

7.5 

372.9 

5.1 

T6 

S6 

286.4 

6.2 

373.7 

4.8 

T7 

S7 

283.8 

7.7 

371.9 

5.4 

T8 

S13 

286.1 

7.0 

373.8 

3.8 

B8 

S11 

285.6 

5.8 

372.5 

5.1 

B7 

S14 

279.1 

6.7 

367.8 

5.5 

B6 

S10 

285.8 

6.8 

371.6 

4.0 

B5 

S1 

287.9 

6.1 

374.5 

3.7 

B4 

S1 

287.7 

6.2 

372.8 

4.4 

B3 

S10 

285.7 

6.5 

372.0 

4.5 

B2 

S3 

280.9 

7.2 

369.3 

4.7 

B1 

S8 

280.9 

6.6 

366.6 

4.7 


Table 8 

Overall 7th-grade Student EOG Mathematics Averages 

2003-2007 


group 

Average 

7th grade 

Math 

EOG 

Scale 

Score 03- 

06 

7th grade 
Math EOG 
Standard 
Deviation 
03-06 

Average 

7th grade 

Math 

EOG 

Scale 

Score 06- 

07 

7th grade 
Math EOG 
Standard 
Deviation 
06-07 

TOP 

284.5 

7.59 

371.7 

5.49 

BOTTOM 

284.0 

7.23 

370.8 

5.20 
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Despite variation between teachers on previous student math test scores, there is little overall 
difference between the incoming student test scores of top and bottom teachers. Figure 6 
compares the distribution of previous 7* grade mathematics scale scores for the top and bottom 
teachers. Both distributions show the same slightly skewed left pattern of scores with a similar 
mean and standard deviation. 


Figure 6 

Previous 7‘** Grade Mathematics EOG Scale Scores 

2006-07 



Notes: Top: Mean 371.7 Standard Deviation 5.49 Median 372 

Bottom: Mean 370.8 Standard Deviation 5.20 Median 371 
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Algebra I Academic Achievement 

The overall pereentage of students seoring profieient at Level III or Level IV on the Algebra I 
EOC varied from 97% to 100% among all top and bottom teachers. The difference between top 
and bottom teachers’ student performance only becomes evident by studying performance at 
Level IV. The percentage of students scoring at Level IV ranged from 86% to 100% for top 
teachers and from 65% to 98% for bottom teachers. There was also a difference between the 
percentage of students scoring at the top half of Level IV. The top teachers percentages ranged 
from 22% to 64% and the bottom teachers from 5% to 38%. These percentages are shown in 
Table 9. 


Table 9 

Algebra I EOC Results by Teacher 
2003-07 


teacher 

school 

% At Level 
III or IV on 
Algebra I 
EOC 

% At Level 
IV on 
Algebra 1 
EOC 

% At High 
Level IV on 
Algebra 1 
EOC 

T1 

S4 

99% 

88% 

22% 

T2 

S9 

100% 

100% 

64% 

T3 

S12 

100% 

99% 

44% 

T4 

S5 

99% 

86% 

27% 

T5 

S2 

100% 

98% 

39% 

T6 

S6 

100% 

98% 

42% 

T7 

S7 

99% 

92% 

39% 

T8 

S13 

100% 

98% 

48% 

B8 

S11 

99% 

98% 

38% 

B7 

S14 

99% 

80% 

11% 

B6 

S10 

100% 

97% 

27% 

B5 

S1 

100% 

94% 

35% 

B4 

S1 

100% 

97% 

23% 

B3 

S10 

99% 

91% 

27% 

B2 

S3 

97% 

67% 

14% 

B1 

S8 

99% 

65% 

5% 


Overall 99.6% of students in top teachers’ classes scored proficient compared to 99.1% in 
bottom teachers classes. At Level IV, there were 95% of top teachers’ students compared to 
86% of bottom teacher students. At the top half of Level IV, the comparison percentages were 
52% to 27%. The distribution of Algebra I scale scores was skewed left for both top and bottom 
teachers, but the mean of the top distribution (167.6) was 3.9 scale points higher than the bottom 
mean of 163.7 even though the previous l'^ grade tests were comparably the same (Figure 7). 
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Figure 7 

Algebra I EOC Scale Scores 
2007-08 



■ TOP 

■ BOTTOM 


Algebra I Scale Scores 


Notes: Top: Mean 167.6 Median 168 

Bottom: Mean 163.7 Median 164 


TEACHERS 

In addition to the achievement described above, the 16 teachers in the full study were also 
surveyed on a variety of topics. The next section of the report focuses on those survey data and 
the difference between top and bottom teachers. 

Survey Results - Educational Preparation 

Five of eight top middle school teachers and six of eight bottom teachers held only a Bachelor’s 
degree in mathematics or mathematics education. Three of eight top and two of eight bottom 
teachers also had an advanced degree in mathematics or education. Three of eight top and two of 
eight bottom teachers were nationally board certified. Not only was there no difference between 
top and bottom middle school teachers with respect to education, but the high school Algebra I 
teachers from the previous study (Haynie , 2008) reported the same levels of educational 
preparation. See Appendix A for the details by teacher. 
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Survey Results - Teaching Experience 

The number of overall years of teaching experience ranged from 10 to 36 years for top teachers 
and from 7 to 37 years for bottom teachers. Seventy-five percent of top teachers had more than 
20 years of teaching experience compared to 38% of bottom teachers. In teaching Algebra I, the 
years of experience ranged from 4 to 30 years for top teachers and from 4 to 24 years for bottom 
teachers. Most teachers, top and bottom, had less than 21 years of teaching Algebra I (63% of 
top teachers and 88% of bottom teachers). 

Compared to the high school results (Haynie, 2008), middle school Algebra I teachers had more 
overall teaching experience than high school Algebra I teachers, with top teachers more 
experienced than bottom teachers in both middle and high. Teachers in the study had less 
Algebra I teaching experience than their overall teaching experience, with middle school 
teachers more experienced than high school teachers, and top teachers more experienced than 
bottom teachers. Despite the differences in experience, there were teachers with only the 
minimum of four consecutive years in all four groups top and bottom (see Figures 8 and 9). See 
Appendix A for the details by teacher. 


Figure 8 

Overall Teaching Experience (Years) 
Middle and High School Teachers 


100% n 
80 % - 
60 % - 
40 % - 
20 % 



75% 



38% 



22 % 


\J /o ~ 

Middle School 
Top Teachers 

Middle School 
Bottom 
Teachers 

High School 
Top Teachers 

High School 
Bottom 
Teachers 

■ >30 

3 

1 

1 

0 

■ 26-30 

2 

0 

1 

0 

■ 21-25 

1 

2 

0 

0 

■ 16-20 

1 

1 

3 

2 

■ 11-15 

0 

2 

1 

1 

■ 4-10 

1 

2 

3 

6 


Notes: Blue represents more than 20 years of experienee. 
Green represents 20 or less years of experience. 
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Figure 9 

Algebra I Teaching Experience (Years) 
Middle and High School Teachers 
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Top Teachers 
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0 
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0 
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0 
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1 
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2 
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Notes: Blue represents more than 20 years of experience. 

Green represents 20 or less years of experience. 

Survey Results - Teacher Planning 

In addition to asking about experience levels, the survey also asked teachers about how they plan 
for instruction. Five of eight top teachers and seven of eight bottom teachers reported that they 
had shared planning time with colleagues. In the high school study, which was conducted before 
the WCPSS districtwide implementation of professional learning communities (PLCs), eight out 
of nine top teachers reported shared planning, but six of nine bottom teachers planned alone. 

Top and bottom middle school teachers agreed that the main use of the shared planning time was 
for lesson planning. Five of eight top teachers marked lesson planning as the top use of time and 
two more top teachers chose lesson planning second. Seven of eight bottom teachers chose 
lesson planning first. 

The high school teachers had marked pacing as their top use of shared planning time. Since all 
but one middle school teacher reported using most to all of the WCPSS Instructional Calendar, it 
was concluded that middle school teachers used the pace set by the calendar. There was less 
concern about how to remediate prerequisites with the well-prepared middle school algebra 
students. High school teachers spent much planning time organizing the curriculum into chu nk s 
that would remediate within new material. The top high school teachers reported using a 
spiraling method of delivery (Haynie, 2008). 


22 


Middle School Algebra 


E&R Report No. 08.22 


Middle school teachers, on the other hand, presented material with a more linear approach that 
spent little time on remediating prerequisites. When asked to rank the importance of Algebra I 
topics defined by the names of chapters in an Algebra I book, top middle school teachers ranked 
the “basics of algebra” last of 14 topics and bottom teachers ranked the basics 10*. Top high 
school teachers had ranked the basics third and the bottom high school teachers second. Besides 
the “basics”, top middle school teachers found the task of ranking the topics to be a difficult one. 
Two top teachers stated that all topics were of equal importance. Using the average rankings, 7 
of 14 topics were ranked in the top five in importance. Since several of the topics from the 
algebra book are not in the state curriculum, it can be concluded that middle school algebra 
teachers spend time on topics not in the state curriculum. 

The top middle school teachers ranked “solving equations” first as did top and bottom high 
school teachers. Bottom middle school teachers ranked it third overall. A topic of disagreement 
was the importance of “linear regression”, a topic that is in the state curriculum. Top high school 
teachers ranked it as sixth in importance, but top middle school teachers ranked it twelfth, 
bottom middle school teachers thirteenth as did bottom high school teachers. See Appendix B 
for details of topic rankings. 

In planning, there were more differences between middle school teachers and high school 
teachers than between top and bottom middle school teachers. Middle school teachers used 
planning time to create lesson plans that progressed in a linear fashion throughout the 
curriculum. Top high school teachers spent planning time developing the pacing necessary to 
deliver a spiraling curriculum (Haynie, 2008). The main difference between top and bottom 
middle school teachers was the importance placed on the basics. Top middle school teachers 
ranked this topic last, while bottom middle school teachers ranked four other topics below the 
basics. 

Survey Results - Instructional Delivery 

The survey also asked teachers to divide their instruction into the following eight instructional 
delivery methods by giving the percentage of the total instructional time spent on each activity: 

• lecture, 

• whole group discussion, 

• projects, 

• small groups, 

• technology, 

• testing, labs, or 

• other. 

There were three delivery methods where differences were noted between top and bottom middle 
school teachers. The methods were lecture, whole group discussion, and small groups. 

Top middle school teachers reported using lecture 0% - 30% of their class time, while bottom 
middle school teachers reported using lecture 25%-50% of the time. The median amount of time 
for top teachers was 22.5%, but 30% for bottom teachers. Thirty-eight percent of top middle 
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school teachers reported using leeture at least 30% of the time compared to 75% of the bottom 
middle school teachers. In the high school Algebra I study (Haynie, 2008), high school bottom 
teachers also had a median of 30%, but high school top teachers reported the highest lecture use 
with a median of 40% (Figure 10). 


Figure 10 

Percentage of Instructional Time Used in Lecture 
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Notes: Blue represents 30% or more of instruetional time. 

Green represents less than 30% of instruetional time. 


The use of whole-group diseussion and small groups was also reported higher by top teaehers 
than bottom teachers in middle school. Whole-group discussion had the same range of 10% to 
35% for both groups, but the median was 25% for top teaehers and 17.5% for bottom teachers. 
Seventy-five percent of top middle sehool teaehers reported using whole group diseussion at 
least 20% of the time compared to 50% of the bottom middle school teachers. The range for top 
teaehers’ use of small groups was 10% to 75% with a median of 20%, while the range for bottom 
teachers was 10% to 25% with a median of 15%. Sixty-three percent of top middle school 
teachers reported using small groups at least 20% of the time eompared to 38% of the bottom 
middle sehool teaehers. The medians for the teachers in the high school study (Haynie, 2008) for 
whole-group diseussion were 10% (top teaehers) and 20% (bottom teaehers). For small groups it 
was 15% for top teachers and 7.25% for bottom teaehers (Figures 1 1 and 12). 
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Figure 11 

Percentage of Instructional Time Used in Whole-Group Discussion 
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Notes: Blue represents 20% or more of instruetional time. 

Green represents less than 20% of instruetional time. 


Figure 12 

Percentage of Instructional Time Used in Small Groups 
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Notes: Blue represents 20% or more of instruetional time. 

Green represents less than 20% of instruetional time. 


Interestingly, reported use of projeets was very uneven. Middle sehool teaehers T1 and B8 
reported spending 20% of their time on projeets while no other teaeher spent more than 5% of 
the time on projeets, and nine reported no time on projeets. Teaehers Tl, T5, B5, and B7 
reported 10% to 15% of time on labs. Five top and three bottom teaehers reported no time in 
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labs. The time spent on testing per week varied from 2 to 20 minutes, with an overall average of 
13 minutes. The results for the high school study were similar (Haynie, 2008). There were no 
noted differences between groups of teachers on these three instructional delivery methods. See 
Appendix C for details by teacher. 

Survey Results - Attitude Toward Students 

Both the high school and middle school teacher surveys contained Likert scale statements that 
measured teacher attitude toward students. The responses were converted to a one to four scale 
from one for “strongly disagree” to four for “strongly agree”. A test of significance found that 
the middle school top teachers’ mean responses compared to middle school bottom teachers’ 
mean responses were all significantly higher (showing stronger agreement) for every statement 
but one. All of the top teacher means were above a 3.0, ranging from 3.38 to 4.00. The bottom 
teacher means ranged from 2.63 to 3.38, with two means below a 3.0. The two areas with means 
below a 3.0 were for the statements that “my students have strong prerequisite skills” and “my 
students work hard” (Figure 13). 


Figure 13 

Mean Response to Attitude Toward Student Statements 
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* significantly higher than all other teacher subgroups 
** significantly higher than high school teacher subgroups 
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In looking at these results eompared to those of the high sehool study (Haynie, 2008), it is also of 
interest to observe that the bottom middle sehool teaehers’ attitude means were only signifieantly 
higher than two means of the top high sehool teaehers (“My students set high standards” and 
“My students seek to understand eoneepts”, and two means of the bottom high sehool teaehers 
(“My students set high standards” and “My students have strong prerequisite skills”). Top 
middle sehool teaeher means were signifieantly higher than all high sehool teaeher means. See 
Appendix D for detail by teaeher and p-values. 

Observation Results 

Classroom observations were made of all 16 top and bottom teaehers. Thirteen were observed 
by both the Evaluation and Researeh Administrator and a Currieulum and Instruetion 
Administrator. Of partieular interest were the behaviors observed in most to all top teaehers. It 
is also of interest that these behaviors were also observed in most top high sehool teaehers, but 
they were observed in fewer of the bottom middle sehool and bottom high sehool teaehers. 
Although bottom teaehers exhibited some of these positive behaviors, they were not as pervasive 
or eonsistent. That is, there was more of a eomplete “paekage” in top teaehers. Table 10 
summarizes these observations. 


Table 10 

Top Middle School Teacher Behaviors 


Behavior 

Middle School 
Top Teachers 

Middle School 
Bottom Teachers 

High School 
Top Teachers 

High School 
Bottom Teachers 

High expectations for all students 

8 out of 8 

1 out of 8 

6 out of 6 

3 out of 5 

Mathematics vocabulary appropriately 
used by teachers and students 

8 out of 8 

1 out of 8 

6 out of 6 

5 out of 5 

Teach bell to bell using an invigorated 
pace 

8 out of 8 

2 out of 8 

4 out of 6 

1 out of 5 

Structured classroom management style 
with mutual respect 

8 out of 8 

4 out of 8 

6 out of 6 

2 out of 5 

Classroom culture in which students were 
free to ask questions, contribute, and offer 
explanations 

8 out of 8 

5 out of 8 

6 out of 6 

0 out of 5 

Sustained feedback 

8 out of 8 

5 out of 8 

6 out of 6 

0 out of 5 

Probing Questions 

7 out of 8 

3 out of 8 

6 out of 6 

2 out of 5 

Instruction adjusted appropriately based 
on student needs at the given moment 

7 out of 8 

4 out of 8 

6 out of 6 

3 out of 5 

Good concept development (not individual 
pieces of information) 

7 out of 8 

6 out of 8 

4 out of 6 

0 out of 5 

Variety of activities and frequent tansitions 

6 out of 8 

6 out of 8 

6 out of 6 

2 out of 5 


Behavior Examples 

High Expectations for All Students 

Observing all students on task all of the elass period in tasks that were rigorous and ehallenging 
was taken as an indieator of high expeetations for all students. One top teaeher ehallenged the 
elass to use their own mental mathematies thinking by saying, “You’re not going to make me 
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write all the steps.” Another teaeher was heard to say, “I’m being mean making you spell math 
terms, right?” Teaeher Tl, who had 36% Afriean-Amerioan/Blaek or Hispanie/Latino students, 
expressed amazement that other teaehers had so little minority representation in elasses. This 
teacher was observed including all students equally in class. 

There was anecdotal evidence of mixed expectations of students in seven of eight bottom 
teachers’ classes. Some students were allowed to be off task. Some were observed reading other 
books, having their heads down on desks, talking off-task, and obviously day-dreaming. One 
bottom teacher expressed annoyance toward a student’s question. On asking for volunteers to do 
an exercise, this same teacher ignored this student’s offer to do the exercise. Some teachers were 
observed giving praise to some students, but not to others. In another class, a student was 
retesting during class time, thus missing the current lesson. A bottom teacher, annoyed by off- 
task questions by one student, answered the question, “Why do we have to take Geometry?” by 
replying, “ then suffer through it and never take Geometry again.” The question was off task, but 
the shut-up answer set very low standards for the students. It validated the opinion that 
mathematics is a dreaded required course for many students. 

Mathematics Vocabulary 

The vocabulary used by top teachers and their students was appropriate for mathematical 
discussions. Top teachers had students study definitions of terms. One teacher had a word wall 
prominently displayed in the classroom. Top teachers pointed out the assumptions of properties 
and the restrictions on operations. Students were required to understand terms such as integer, 
radical, nonnegative, etc. to be able to use assumptions and restrictions. Top teachers had 
students read their math book, which led to use of proper terms in classroom discussions such as 
“leading coefficient”, “maximum and minimum of a quadratic function”, “denominator”, 
“exponent’, “base” , and “radical sign”. 

Inappropriate use of vocabulary was observed in seven of the eight bottom teachers’ classes. 
Exercises were given with no instructions. Algorithms were used with no discussion of terms 
involved. Non-mathematical language was used instead of proper terms. Teachers were heard to 
say “flip”, “plug-in”, “send over”, “bottom”, and “over” exclusively instead of “use reciprocal”, 
“substitute”, “subtract from both sides”, “denominator”, and “divided by”. Mathematical terms 
were also used incorrectly. One bottom teacher was heard saying, “Is the negative in (-b/2a) for 
the b or the whole equation?” (-b/2a is an expression not an equation). In another class, a 
student was heard to ask, “How do I solve ((6x+3)/3)?” He should have used simplify. This was 
a missed opportunity for vocabulary instruction that pointed out the difference between an 
expression and an equation. A particularly bad misuse of language was recorded during the 
observation of teacher Bl, who said, “There is an equation that we can use to come up with the 
axis of symmetry point” and “the minimum is the axis of symmetry.” In reality, the axis of 
symmetry is a line, not a point, and the minimum of a parabola is the y-value of the vertex. 

Classroom Culture 

Top teachers in middle school and high school had a classroom culture in which students were 
free to ask questions, contribute, or offer explanations. Observers recorded examples of inquiry. 
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wrong answers, personal ehallenge, eollaboration, and disequilibrium. In one class, there were 
two volunteers for the same problem, and each was asked to put the solution on the board /or 
comparison. Students seemed quite comfortable doing this. They were willing to take the risk 
of being wrong, but also knew that there might be two ways to get the right answer. In accepting 
a student’s answer, a teacher said, “That’s right. Usually I add the 4 and 8 and then subtract the 
7 but you can add in any order that you like.” In another class, the teacher said, “Your group is 
the only one that made a chart. Will you go up and explain it?” The student responded, “Sure. 
For our formula, we looked at the second difference. . .” 

Activities in top teacher classes centered on mathematical understanding, invention, and sense- 
making. Three supporting teacher quotes were heard: 

• “Why do we assume that the variables all represent nonnegative numbers under the radicals?” 
(note: all the radicals were square roots). 

• “Very good! I’m so glad that I asked you since I never thought of it that way.” 

• “What are the restrictions on these rational expressions? Be sure to list them first.” 

Observations of bottom teachers provided, on the other hand, some examples of a less positive 
classroom culture. In one class the teacher’s response to a student asking, “Why is it the vertical 
line test and not the horizontal?”, was “Because it’s the vertical line test”. Student “why” 
questions were answered with statement such as, “This is the way its done, remember your 
properties, and the steps.” In another class, an open-ended question was accepted with only one 
answer, and a student who wanted to offer another solution was turned away. 

Structured Classroom Management 

In top teachers’ classes, there were organized classroom routines. Students clearly understood 
their roles. All class time was used on worthwhile mathematical tasks that kept all students 
engaged. An invigorated pace was used with tightly organized lessons/activities that were 
broken into manageable sections. There were clearly stated daily objectives. 

Within this structure, teachers were able to capitalize on unexpected learning opportunities and 
adjust instruction appropriately to meet student needs. Teachers listened carefully and used 
formative assessments regularly. In one of the observations of teacher TI, the lesson for the day 
was put aside while more work was done on the homework task because many students had 
questions. On the other hand, one of the bottom teachers was observed repeating an entire lesson 
as the student who asked a question was saying, “I only wanted one example”, and the other 
students were restless and inattentive. 

One top teacher taught students to be mathematicians using a repetitive instructional exploration 
rubric with each new topic. This teacher also explained upcoming test grading rubrics as quizzes 
were marked by students in class, thus enabling students to take ownership of their learning. 
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Sustained Feedback 

Top teachers were persistent in giving, revisiting, and adjusting feedback to students. One 
teacher said, “I went back and did some problems at home last night and the slide method does 
not always work, let me show you”. Other quotations were recorded: 

• “You’ve got some mistakes in there, try again”. 

• “I should see better things on the scientific notation than I saw on the homework”. 

• “I think what you meant to do was pull out the GCF. Will you come up and fix that?” 

• “[student’s name] had a good idea of building a smaller model and working with smaller 
numbers.” 

Probing Questions 

Top teachers asked probing questions of students. Examples of some probing questions recorded 
during observations included the following: 

• “What is another way we could solve this?” 

• “How do we know she has the right solution?” 

• “How many of you have a different approach? We would like to see it.” 

• “How many ways are there to approach these problems?” 

• “Which one of these explanations helped most?” 

Focus Group Results 

As in the focus groups of previous effectiveness studies, the main difference between the top and 
bottom groups was what each group emphasized in their discussions. The top teachers spent 
most focus group time discussing and sharing how they teach Algebra I. The bottom teachers 
spent much more time on the negatives of misplaced students. Their responses were more of the 
“poor me, what can I do” variety, while top teachers wanted to hear and share what they and 
their colleagues did. 

Top teachers described their students as motivated, conscientious, going above and beyond, 
thirsting for learning, perfectionists, delightful, and fun to teach. Top teachers concluded that 
middle school Algebra I is a challenging course that works well for students who should be 
there. Bottom teachers described their students as the sharpest, the brightest, good students, hard 
workers, but, not all are motivated. They said that some students are not willing to dig and do 
work on new topics. They also said that some students are placed in Algebra I by a parent 
regardless of whether the teachers think the child is ready and “drag down” the class. 

CONCLUSIONS 

The main goals of this research study were to collect system-specific data to help teachers and 
district leadership understand current middle school Algebra I practices, identify and share best 
instructional practices in middle school Algebra I, continue to build a series of studies that 
identify the role of teachers, and other system staff/departments in the school improvement 
process, and identify the practices of effective improvement. 
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In 2007-08, 28 WCPSS middle schools offered Algebra I and 97% of these students scored 
proficient on the Algebra I EOC at the end of the course. The 29% of Algebra I students that 
take algebra before high school are among the strongest students in the district, yet only 79% 
scored at Level IV. The percentage of middle schools that made expected growth with their 
Algebra I students was 79%, with 68% making high growth. Despite high proficiency rates, 
there is ample room for improvement in student performance of middle school Algebra I 
students. 

The average teacher residual scores, utilized to form the top and bottom groups for this study, 
identified teachers who could be used to document best instructional practices in middle school 
Algebra I. The results indicated that there was a qualitative difference between the performances 
of students in top versus bottom teacher classes. The difference in the average grade 
mathematics scale scores of those two groups was negligible, but the difference in Algebra I 
EOC scale scores at the end of the course was not. 

During focus-group interviews, the teachers stated that student success in high school 
mathematics was another indicator of teacher success in middle school Algebra I. They said that 
taking calculus in high school was the goal of most middle school algebra students. This 
statement and the difference in the quality of the results between top and bottom middle school 
teachers led to the question if there is a difference in the course-taking patterns of the top and 
bottom teachers. In order to begin to explore this question, a follow-up cohort study analyzed 
the high school mathematics course-taking patterns of students of the top and bottom teacher of 
this study. It found that for these two teachers and this cohort of students, there was a difference 
in course selection and the percentage of students who studied mathematics at the highest level. 
Sixty-nine percent of top teacher TI’s students took the AP Calculus or higher courses in 12**' 
grade compared to only 19% of bottom teacher BI’s students. It needs to be noted that there are 
many possible intervening factors that would add to the explanation of the results, and additional 
study is recommended (Haynie, 2009). 

All middle school Algebra I teachers had a positive attitude overall toward their students and 
their teaching assignment; yet top middle school teachers were significantly more positive than 
bottom middle school teachers. Bottom teachers were distracted by the students who were 
misplaced and lacked the math prerequisites or the study skills necessary for highest 
performance. Top teachers focused on the positive qualities of each student, expecting all 
students to rise to their high expectations. 

NCTM’s (2000) Principles and Standards for School Mathematics sets the teaching standards 
that all mathematics teachers K-I2 should strive to hold in their classrooms. The top teachers in 
this study exemplified many of these standards in their practice. Top teachers used open-ended 
questions and explorations. There appeared to be no fear of what students might offer as 
answers. It was acceptable for students to be “smarter” and “quicker” in math than the teacher. 
Top teachers were confident in the role of facilitator and also shared in the class as a participant 
in learning. Topics were developed using organized, repetitive, and clear concept development. 
Concept development was achieved by using probing questions and sustained feedback. These 
findings were consistent with the NCTM stance: “Without connections students must learn and 
remember too many isolated concepts and skills” (NCTM, 2000, p.275), and “Teachers should 
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help students recognize that all mathematics can and should be understood” (NCTM, 2000, 
p.125). 

Top teachers used a variety of instructional methods. There was less lecture and more use of 
whole group discussion and small groups than in bottom teacher classes. Overall there was more 
student ownership of their learning. Top teachers used structured classroom management that 
provided students with clear definitions of their roles. Top teachers were observed adjusting 
their instruction appropriately by using frequent formative assessment. These findings also echo 
the NCTM stance: “Teachers can understand students’ thinking when they listen carefully to 
students’ explanations” (NCTM, 2000, p.l26). 

Top teachers assumed knowledge of basic algebraic skills and taught an enrichment-filled course 
at an invigorated pace. Students were exposed to both curriculum and beyond-curriculum topics. 
Precise mathematical vocabulary was expected and modeled by the teacher and all students, 
which again dovetails with the NCTM principals and standards: “Teachers should model 
appropriate conventional vocabulary” (NCTM, 2000, p.l31). “Students are expected to learn 
serious, substantive mathematics with an emphasis on thoughtful engagement and meaningful 
language” (NCTM, 2000, p.213). 

DISCUSSION AND IMPLICATIONS 

Green (2005) in his book Expectations: How Teacher Expectations Can Increase Student 
Achievement and Assist in Closing the Achievement Gap discusses at length what research says 
about the correlation between teacher expectations and student achievement. The contrast 
between observations and the survey attitude items of the top and bottom middle school teachers 
of this study provided support in WCPSS of this research finding. Many educators and parents 
assume that middle school algebra students are the brightest and best. The top and bottom 
teachers of this study concurred; they were proud to teach such a high level course in middle 
school. Yet top teachers in this study held a significantly higher expectation for all their students 
than did the bottom teachers. During observations it was clear that students understood this 
expectation and had made it their own. The students of the top teachers fared better in terms of 
achievement during their year in Algebra I as measured by the EOC. A cohort study of high 
school mathematics course-taking patterns of students implied that they also rose to higher 
challenges later on in high school (Haynie, 2009). 

The newly adopted WCPSS Board of Education (BOE) Goal is one that centers on high 
academic growth: 

WCPSS students will demonstrate high academic growth; by 2014, all 

students will graduate on-time prepared to compete globally. 

In addition, the BOE has identified middle school leading indicators to be monitored in 
pursuit of that long-term goal. One of the indicators is to increase the enrollment and 
proficiency in Algebra I. As this study shows, proficiency in Algebra I is already very 
high, but it is less clear that the numbers of students should be increased. Proficiency in 
Algebra I can be reached without reaching the goal of high growth. 
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This study has identified behaviors that ean be studied and adopted by all middle sehool 
teaehers that will make improvements in their elassroom instruetion Yet attitude may be 
the most difficult to address in staff development activities. It is hoped that uncovering 
this difference will help to facilitate self-reflection that will lead to change. Middle 
school algebra teachers cannot rest on the high performance of their students as a group. 

They must also strive for the highest growth for all. 

The following recommendations for improvement are made to all middle school Algebra I 
teachers: 

• BELIEVE that all students will work and succeed at a very high level, and communicate that 
belief to all students. 

• Study and use appropriate Algebra I vocabulary. 

• Locus on concept development throughout topics and avoid isolated memory tricks. 

• Listen carefully to students and give meaningful feedback in a positive way. 

• Encourage all students to participate by having whole group discussions and small group 
experiences. 

• Support a classroom culture that promotes open discourse between teacher and students. 

• Study residual rosters. 

• Study mathematics to increase depth of content knowledge. 

• Reflect on growth data as well as performance data and instructional practice. 

The following recommendations are made to school-based leadership: 

• Support a school culture where all students can be successful (high expectations for all 
students). 

• Support a school culture that promotes open discourse between school leadership and staff 

• Share results data with teachers including effectiveness rosters and indices. 

• Use purposeful teacher course assignment. 

• Reflect on growth data as well as performance data and instructional practice. 

The following recommendations are made to district leadership: 

• Make observations of most effective teachers that can be shared districtwide. 

• Eacilitate teacher reflection on teacher held beliefs and attitudes. 

• Eacilitate the study and appropriate use of Algebra I vocabulary. 

• Help teachers focus on concept development throughout topics and avoid isolated memory 
tricks. 

• Support a district culture where all students can be successful (high expectations for all 
students). 

• Support a district culture that promotes open discourse between school leadership and staff. 

• Support teacher improvement efforts. 

• Provide data to teachers and schools on their effectiveness. 

• Provide growth data as well as performance data to teachers. 
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Appendix A 

Educational Preparation and Teaching Experience 
Middle School Teachers 


Teacher 

School 

Years 

Experience 

Teaching 

Years 

Teaching 

Algebra 

Years 

Teaching 

Middle 

School 

Algebra 

Bachelors 
In Math or 
Math 

Education 

Advanced 
Degree in 
Math or 
Education 

Nationally 

Board 

Certified 

T1 

S4 

24 

21 

21 

n 

n 

n 

T2 

S9 

28 

7 

7 

y 

n 

y 

T3 

S12 

10 

4 

4 

n 

n 

n 

T4 

S5 

26 

11 

8 

n 

n 

y 

T5 

S2 

16 

4.5 

4 

y 

n 

n 

T6 

S6 

34 

30 

30 

y 

y 

n 

T7 

S7 

36 

36 

13 

y 

y 

n 

T8 

S13 

33 

19 

19 

y 

y 

y 

B8 

S11 

10 

8 

8 

y 

n 

n 

B7 

S14 

37 

24 

22 

n 

n 

y 

B6 

S10 

23 

8 

8 

n 

n 

n 

B5 

S1 

22 

17 

9 

y 

y 

n 

B4 

S1 

14 

6 

6 

y 

n 

y 

B3 

S10 

11 

11 

11 

y 

n 

n 

B2 

S3 

7 

4 

4 

y 

n 

n 

B1 

S8 

19 

9 

7 

y 

y 
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Appendix B 
Table B1 

Median Ranking of Algebra Topics 



TOPIC 

MEDIAN 
MIDDLE 
SCHOOL TOP 
TEACHERS 

MEDIAN 

MIDDLE 

SCHOOL 

BOTTOM 

TEACHERS 

MEDIAN HIGH 
SCHOOL TOP 
TEACHERS 

MEDIAN 

HIGH 

SCHOOL 

BOTTOM 

TEACHERS 

Overall Median 

1 

Basics of Algebra (order of 
operations, rules of 
exponents, properties 
operations with real 
numbers, etc.) 

13.0 

9.0 

2.5 

2.0 

4.0 

2 

Solving equations 

2.0 

3.5 

1.5 

1.5 


3 

Solving inequalities 

7.0 

6.0 

10.0 

6.0 

6.5 

■ 

Solving and applying 
proportions 

12.5 

10.0 

12.5 

7.5 

11.0 

5 

Graphing functions 

6.0 

4.0 

4.5 

4.5 

5.0 

6 

Functions 

5.5 

3.0 

5.0 

5.5 

5.0 

7 

Linear equations and their 
graphs 

4.0 

3.0 

2.0 

2.5 

3.0 

8 

Systems of equations and 
inequalities 

3.5 

6.5 

7.0 

4.5 

6.0 

9 

Exponents and exponential 
equations 

9.0 

8.0 

8.5 

9.5 

9.0 

10 

Polynomials and factoring 

6.0 

7.0 

7.0 

6.5 

7.0 

11 

Quadratic equations and 
functions 

6.0 

8.0 

9.0 

10.0 

9.0 

12 

Radical expressions and 
equations 

9.5 

9.0 

12.0 

11.5 

11.0 

13 

Rational expression and 
functions 

11.5 

13.0 

13.0 

13.0 

13.0 

14 

Linear regression and data 
analysis 

12.5 

12.0 

6.0 

12.5 

12.0 
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Table B2 

Rankings of Algebra Topics 


Topic 

Number 

TOPIC 

RANK-MEDIAN 
MIDDLE 
SCHOOL TOP 
TEACHERS 

RANK- 

MEDIAN 

MIDDLE 

SCHOOL 

BOTTOM 

TEACHERS 

RANK- 

MEDIAN 

HIGH 

SCHOOL 

TOP 

TEACHERS 

RANK- 

MEDIAN 

HIGH 

SCHOOL 

BOTTOM 

TEACHERS 

RANK- 

Overall 

Median 

2 

Solving equations 

1 

3 

1 

1 

1 

8 

Systems of equations and 
inequalities 

2 

6 

7 

4 

6 

7 

Linear equations and their 
graphs 

3 

1 

2 

3 

2 

6 

Functions 

4 

1 

5 

6 

4 

5 

Graphing functions 

5 

4 

4 

4 

4 

10 

Polynomials and factoring 

5 

7 

7 

8 

8 

11 

Quadratic equations and 
functions 

5 

8 

10 

11 

9 

3 

Solving inequalities 

8 

5 

11 

7 

7 

9 

Exponents and exponential 
equations 

9 

8 

9 

10 

9 

12 

Radical expressions and 
equations 

10 

10 

12 

12 

11 

13 

Rational expression and 
functions 

11 

14 

14 

14 

14 

4 

Solving and applying 
proportions 

12 

12 

13 

9 

11 

14 

Linear regression and data 
analysis 

12 

13 

6 

13 

13 

1 

Basics of Algebra (order of 
operations, rales of 
exponents, properties 
operations with real 
numbers, etc.) 

14 

10 

3 

2 

3 
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Appendix C 

Percentage of Instructional Time by Delivery Method 


Teacher 

School 

Lecture 

Whole Group 
Discussion 

Projects 


Technology 

Testing 

Labs 

Other 

T1 

S4 

10 

10 

20 


10 

10 

10 

0 

T2 

S9 

10 

35 

5 


20 

10 

0 

0 

T3 

S12 

20 

30 

0 


10 

20 

0 

0 

T4 

S5 

30 

25 

0 

25 

0 

20 

0 

0 

T5 

S2 

30 

20 

0 

10 

15 

15 

10 

0 

T6 

S6 

0 

10 

5 

75 

5 

5 

0 

0 

T7 

S7 

25 

25 

0 

10 

30 

10 

0 

0 

T8 

S13 

30 

30 

5 

10 

10 

10 

5 


B8 

S11 

25 

10 

20 

20 

10 

15 

0 

0 

B7 

S14 

50 

15 

0 

15 

5 

5 

10 

0 

B6 

S10 

45 

10 

0 

15 

10 

20 

0 

0 

B5 

S1 

25 

15 

0 

15 

15 

15 

15 

0 

B4 

S1 

30 

35 

1 

20 

10 

2 

2 

0 

B3 

S10 

40 

20 

0 

10 

10 

20 

0 

0 

B2 

S3 

30 

25 

0 

25 

5 

10 

5 

0 

B1 

S8 

30 

30 

5 

10 

35 

20 

5 

0 


40 











































































































































































Middle School Algebra 


E&R Report No. 08.22 


Appendix D 
Table D1 

Teacher Attitude Responses 


Teacher 

School 

My 

students 
work hard 

My 

students 
do their 
homework 

My 

students 
seek extra 
help 

My 

students 
set high 
standards 
for 

success 

My 

students 
have strong 
prerequisite 
skills 

My 

students 
seek to 
understand 
the 

concepts of 
algebra 

T1 

S4 

SA 

SA 

SA 

SA 

A 

SA 

T2 

S9 

SA 

SA 

SA 

SA 

SA 

SA 

T3 

S12 

SA 

SA 

SA 

SA 

A 

SA 

T4 

S5 

A 

A 

SA 

SA 

A 

A 

T5 

S2 

SA 

SA 

SA 

SA 

SA 

SA 

T6 

S6 

SA 

A 

SA 

SA 

SA 

SA 

T7 

S7 

SA 

SA 

A 

SA 

D 

SA 

T8 

S13 

A 

SA 

SA 

SA 

SA 

A 

B8 

S11 

A 

SA 

A 

SA 

D 

A 

B7 

S14 

A 

SA 

SA 

A 

A 

A 

B6 

S10 

A 

A 

A 

A 

A 

A 

B5 

SI 

A 

A 

A 

A 

A 

A 

B4 

SI 

A 

A 

SA 

SA 

SA 

SA 

B3 

S10 

A 

A 

SA 

A 

A 

A 

B2 

S3 

D 

D 

A 

D 

SD 

A 

B1 

S8 

D 

A 

A 

A 

D 

A 


Table D2 
p-values 

2-sample t-test of difference 



Middle 

Top 

/Middle 

Bottom 

Middle 

Top 

/High 

Top 

Middle 

Bottom 

/High 

Top 

Middle 

Bottom 

/High 

Bottom 

High 

Top/ 

High 

Bottom 

1 . My students work hard 

** 

** 




2. My students do their homework 

* 

** 




3. My students seek extra help 

* 

** 




4. My students set high standards 


** 

★ 

★ 


5. My students have strong prerequisite skills 


** 


* 

* 

6. My students seek to understand concepts 

** 

** 

** 




* significant at the .05 level 
** significant at the .01 level 
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